Skip to content

Conversation

aonnikov
Copy link
Member

No description provided.

@aonnikov aonnikov requested a review from lexiv0re May 26, 2025 06:57
Copy link

Connected to Huly®: UBERF-11179

lexiv0re
lexiv0re previously approved these changes May 26, 2025
@aonnikov aonnikov marked this pull request as draft May 30, 2025 02:47
@aonnikov
Copy link
Member Author

Migrated manually, not needed for now

@aonnikov aonnikov closed this Jun 24, 2025
@aonnikov aonnikov reopened this Jul 16, 2025
@aonnikov aonnikov force-pushed the fix-datalake-workspace-migration branch 2 times, most recently from a7f866b to 3c6ba8c Compare July 17, 2025 09:36
@lexiv0re
Copy link
Collaborator

lexiv0re commented Aug 4, 2025

@aonnikov what guarantees the existence of global_account namespace in the DB before the migration is run? For this there must be some kind of dependency that the account's migrations are completed.

@aonnikov
Copy link
Member Author

aonnikov commented Aug 4, 2025

Do you think it is needed? We can do this by waiting for account service to start, but I don't think it changes the situation much.
There are three scenarios:

  1. global_account already exists.
  2. global_account does not exist but will be created soon - datalake will fail on migration, restart, and after few restarts will finish the migration. Probably we need to do better error handling here.
  3. global_account does not exist and won't be created because account is in MongoDB or somewhere else. As far as I remember, we decided to not support this scenario.

@lexiv0re
Copy link
Collaborator

lexiv0re commented Aug 5, 2025

Yes, I was concerned about 2. Won't there be a race condition when global_account is created already but workspaces migration is not finished yet and datalake might run its migration on partial data with some workspaces info missing?
3. yes, we decided not to support

@aonnikov
Copy link
Member Author

Yes, I was concerned about 2. Won't there be a race condition when global_account is created already but workspaces migration is not finished yet and datalake might run its migration on partial data with some workspaces info missing? 3. yes, we decided not to support

If we wait for account service to be ready before starting datalake, will it work? I pushed changes that try to perform some /rpc to ensure the account service started.

@aonnikov aonnikov marked this pull request as ready for review August 11, 2025 06:09
@aonnikov aonnikov marked this pull request as draft August 11, 2025 06:11
@lexiv0re
Copy link
Collaborator

Yes, if we wait for any rpc from account that'd guarantee that accounts' migrations are completed. Did you push the changes here? I cannot see them :)

@aonnikov aonnikov force-pushed the fix-datalake-workspace-migration branch 4 times, most recently from be5091e to 3c830c4 Compare August 11, 2025 13:35
Signed-off-by: Alexander Onnikov <[email protected]>
@aonnikov aonnikov force-pushed the fix-datalake-workspace-migration branch from 3c830c4 to 3f86fd2 Compare August 11, 2025 16:04
@aonnikov
Copy link
Member Author

Yep, pushed the changes

@aonnikov aonnikov marked this pull request as ready for review August 12, 2025 04:05
@aonnikov aonnikov merged commit 8d9c84b into develop Aug 12, 2025
23 of 24 checks passed
@aonnikov aonnikov deleted the fix-datalake-workspace-migration branch August 12, 2025 04:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants